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ABSTRACT 

We present HST NICMOS+ACS and Spitzer IRAC+MIPS observations of 41 galaxies at 2 < z < 3.5 
in the FIRES MS 1054 field with red and blue rest-frame optical colors. About half of the galaxies are 
very compact (effective radii rg < 1 kpc) at rest-frame optical wavelengths, the others are extended 
(1 < Tg < 10 kpc). For reference, 1 kpc corresponds to 0'.'12 at z = 2.5 in the adopted cosmology. 
We separate actively star forming galaxies from quiescent galaxies by modeling their rest-frame UV- 
NIR SEDs. The star forming galaxies span the full range of sizes, while the quiescent galaxies all 
have Ve < 2kpc. In the redshift range where MIPS 24/im imaging is a sensitive probe of re-radiated 
dust emission {z < 2.5), the 24/im fluxes confirm that the light of the small quiescent galaxies is 
dominated by old stars, rather than dust-enshrouded star formation or AGN activity. The inferred 
surface mass densities and velocity dispersions for the quiescent galaxies are very high compared to 
those in local galaxies. The galaxies follow a Kormendy relation (between surface brightness and size) 
with approximately the same slope as locally, but shifted to brighter surface brightnesses, consistent 
with a mean stellar formation redshift of Zf ~ 5. This paper demonstrates a direct relation between 
star formation activity and size at z ~ 2.5, and the existence of a significant population of massive, 
extremely dense, old stellar systems without readily identifiable counterparts in the local universe. 
Subject headings: galaxies: formation — galaxies: evolution — galaxies: high redshift — galaxies: 
fundamental parameters — galaxies: structure — infrared: galaxies 
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1. INTRODUCTION 

In the local universe, galaxy structure correlates with 
physical properties such as star formation rate and his- 
tory, mass, environment, gas and dust content, metallic- 
ity etc. This is likely a consequence of the fact that star 
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formation activity scales with environ ment and structure 
(bulge to disk ratio) scales with mass (|Kauffmann et ahl 
2004). High density environments are dominated by 
early type galaxies which are homogeneously old, red, 
quiescent and with low dust contents, while lower density 
environments are dominated by late type galaxies which 
on average are bluer, less massive and show a range of 
ages, dust contents, star formation histories, me tallicities 
etc. fe.g. iGoto eral]l2004l : iBaldrv et al.ll2006[ ). Obser- 
vations suggests that these correlations exist out to at 
least z ~ 1.4, where clusters are still dominated by uni- 
formly large, old, red early type galaxies with little ongo- 
ing star formation and dust, although the fraction of blue 
late type g:alax i es in clusters increase with redshift (e.g; 
iBlakeslee et al." '2003; 'Toft et al.' l2003; "Bell et al.' '2 0041: 
Mcintosh et al. 2005; Lidman et al. 2004; Mulhs e tld] 
[20051 : ICooper et all 120071 : [Cassata et al. 2007). Under- 
standing the epoch and nature of the onset of these rela- 
tions are key issues for understanding galaxy formation, 
and it is essential to push simultaneous studies of struc- 
ture and star formation properties of galaxies to higher 
redshifts. 

Until recently, most work on high redshift galaxies was 
done using rest-frame UV selected Lyman break galax- 
ies, which were thought to be representative of the high 
redshift galaxy population. An HST study comparing 
the optical (rest-frame UV) WFPC2 and NIR (rest-frame 
optical) NICMOS structure of Lyman break galaxies in 
the Hubble D eep Field North (HDFN) showed that these 
were similar (' Dickinsonl l2000f ) . This result discouraged 
further NICMOS observations of high redshift galaxies 
in the following years, since NICMOS observations are 
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much less efficient tiian optical WFPC2 an d ACS obser- 
vations. In a number of studies i Giavalis co et al.lll996l: 
iLowenthal et all Il997l; lDickinsonn2QQQi : iFerguson et all 
|2004; Bouwe ns et alJ 2QQ4f ) it was assumed that rest- 
frame UV structural properties (such as Hubble-type and 
size) were representative of the rest-frame optical struc- 
tural properties (which traces the bulk of the stellar mass 
distribution, and is related to the dynamical state of 
the galaxies). The similarity of the rest-frame optical 
and UV structural properties of high redshift galaxies in 
the HDFN was later confirmed in a more detailed study, 
which calculated an internal UV-optical color dispersion, 
and sho wed that this quantity was small for galaxies at 
z - 2.3 (|Papovich et a l. 2005). 

The Faint InfraRed Extragalactic Survey (FIRES, 
iLabbe et al.l [2003; Franx et al. 2003) discovered a pop- 
ulation of distant (2 < z < 3.5) red galaxies (DRGs) 
using the simple selection criterion J — Ks > 2.3, which 
proved very efficient for locating massive evolved galax- 
ies and massive galaxies with significant amounts of dust 
and ongoing star formation at these redshifts. Since 
then a number of studies have employed this selection 
technique to study different aspects of the properties of 
DRGs, and their relation to other high redshift popula- 
tions of galaxies. Based on optical-NIR SED fits it has 
been demonstrated that DRGs on average have higher 
stellar masses, higher mass-to-light ratios, more dust 
and higher star formation rates (^Forster Schreiber et al.l 
l2004l ) than the rest-frame UV selected Lyman Break 
Galaxies (LBGs). DRGs dominate in stellar mass lim- 
ited (M > lO^^M©) samples (van Dokkum et al. 2006) 
and the brig ht end of the rest-fram e optical luminos- 
ity function (Marchesini et all 1200 7^ over galaxies with 
J — Ks < 2.3 in the same redshift range (for con- 
venience we label these galaxies "distant blue galax- 
ies" (DBG^) in the^ following) and they cluster more 
strongly (|Quadri et al.l 120071 ). DRGs also dominate 
the stellar mass density in luminosity selected samples 
(|R.udnick et all l200d 120061 ). over DBGs. DBGs share 
many properties with LBGs (they are younger, less 
massive and less obscured than DRGs), but there is 
not a one to one correspondence. The majority of of 
LBGs have {J — K < 2.3), but a not insignificant sub - 
set have redder J — K colors (see iQuadri et al.ll2006l ). 
and DBGs are not necessarily LBGs, some of them do 
not meet the UnGR color criterion (Steidel et al. 1996) 
or are too faint i n the R band (|Quadri et al.l 120061 : 
Ivan Dokkum et aP 1200 6). Recently, the addition of mid 
infrared (MIR) observations from the Spitzer Space tele- 
scope has allowed a better separation between DRGs 
which are red due to old passively evolving stellar pop- 
ulations, and D RGs which are red due to dust obscured 
star formation (i Labbe et al.ll2005l: iPapovich et ah 2006. : 
IWebb et al.l f2006: Re ddv et al.ll2006l V 

In the GOODS-S field about 50% of the DRGs (in the 
redshift range 1.5 < z < 2.5 where MIPS is sensitive 
to star formation , see Se c|6]) were detected with MIPS 
at 24/im (Papovic h et al.i r2006). and a simil ar fraction of 
DRGs were detected in the GOODS-N field (|R.eddv et all 
I2OO6). The MIPS data on these fields was not very deep 
(only sensitive to SFR > 100MQyr~^ at z = 2), and 
it was argued, based on other diagnostics of star for- 
mation, that < 10% of the DRGs in the GOODS-S field 
have rest-frame UV-optical-IR colors consistent with qui- 



escent stellar populations ([Papovich et al.l l2006l ). In 
the Extended HDFS, MIPS 24/im observations detected 
65% of the DRGs in the same redshift range, indicating 
that this fraction is dusty actively star forming galax- 
ies ((Webb et al. 2006). Based on their non-detection by 
MIPS at 24/im, the remaining DRGs (35 - 50%) are pre- 
sumably red due to more quiescent stellar populations. 
This picture is broadly consistent with NIR spectroscopy 
of a sample of 20 bright {K < 19.6) DRGs in the red- 
shift range 2 < z < 2.6 which found that a surprisingly 
large fraction (45%) of these had no emission lines, and 
continu um breaks consiste nt with evolved stellar popu- 
lations (|Kriek et al.ll2006cr ). 

Interestingly, DRGs show a difference between their 
rest-frame UV and optical structural properties: In a 
study based on th e Hubble Ultra Deep Field NICMOS 
(Tho mpson et all [2005) and ACS data (|Beckwith et all 
2006), it was demonstrated that the structure of DRGs 
depends strongly on wavelength (|Toft et al.ll2005l ). The 
six DRGs in this field were shown to have regular cen- 
trally symmetric rest-frame optical structure, while the 
rest-frame UV structure, as observed with ACS were 
much more irregular, and thus not representative of the 
underlying stellar mass distribution of the galaxies. This 
study however did not include rest-frame UV selected 
galaxies to test whether this was also the case for them. 

Recently, the existence of a population of extremely 
dense massive galaxies at z > 1.2 have been suggested 
by a number of studies. In the UDF, 2/6 DRGs with 
2 < z < 3.5 were found to be unresolved by NIC- 
MOS Hfiqow band observations (Toft et al. 2005.), cor- 
responding to rest-frame optical physical sizes of Tq < 
0.8/cpc, a factor of ~ 5 smaller than galaxies of similar 
mass in the local universe. In another study of the UDF, 
4/7 spectroscopically confirmed massive (M > 10^^ M©) 
early- type galaxies with 1.4 < z < 2.5 were found to 
have similarly small physical siz es r^, < O.Skpc, m ea- 
sured in the ACS zfsbolp band (|Daddi et al.ll2005l ). In 
a study at lower redshifts (1.2 < z < 1.7), 10 massive 
(-- 5 X 10^^ M0) galaxies in the MUNICS survey were 
shown to be a factor of ~ 4 smaller than local galaxies of 
similar mass. In a NICMOS Hfiqqw band study of the 
HDFS, 5/1 4 DRGs with 2 ^ z < 3.5 were found to have 
Te ^ 1 kpc (|Zirm et al.ll2007l ). This study also suggested 
the existence of correlation between star formation activ- 
ity and size at z ~ 2.5. The rest-frame optical-NIR SEDs 
of the compact (re < Ikpc) DRGs were better fitted by 
single stellar population models without dust, than con- 
stantly star forming model with dust, while the opposite 
was true for the more extended (rg > Ikpc) DRGs. The 
main uncertainties of the above studies are that they are 
based on very small samples, and the possibility that 
AGN may be dominating the light in the compact galax- 
ies, and affect their size measurements, mass and star 
formation estimates. Interrestingly, recent dynamical 
studies of sub-millimeter galaxies (SMGs) at z ~ 2—3 
also imply high matter densities, from large dy namical 
mass e s ^ 10^^ M^t) within radii < 2kpc (T acconi et all 
I2OO6I : iBouche et all 120071 ) In this paper we present a 
study based on deep NICMOS and ACS observations of 
a sample of 27 DRGs and a sample of 14 DBGs in the 
redshift range 2 < z < 3.5 in the FIRES MS1054-03 field 
(Forster Schreiber et ah 2006). In addition to an anal- 
ysis of how the structure of DRGs and DBGs compare 
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to each other and depend on wavelength, we combine 
the ground based optical-NIR FIRES photometric cata- 
log with deep MIR Spitzer Space telescope IRAC pho- 
tometry to construct rest-frame UV-NIR SEDs for the 
galaxies, from which we constrain properties of the stel- 
lar populations (star formation history, stellar mass, dust 
content, etc.) and investigate how these correlate with 
structural properties derived from the HST data. We 
also add deep MIPS 24/im imaging to the analysis, and 
demonstrate how size correlates with 24/im flux. 

Using this much larger sample of galaxies (and span- 
ning a larger range of colors and masses) than previous 
studies, we demonstrate that there is indeed a correla- 
tion between star formation activity and size z ~ 2.5, 
ranging from extremely compact quiescent galaxies to 
extended star forming galaxies. Furthermore, using di- 
agnostics based on X-ray and MIPS 24/im observations 
we independently demonstrate that the small sizes of the 
quiescent galaxies are not caused by AGN activity. 

The paper is organized as follows: In Sec. [2] we de- 
scribe sample selection and the data, including details 
of the HST and Spitzer data reduction and photometry, 
and the construction of optical-MIR photometric cata- 
logs and SED fits. In Secl3]we present the HST images of 
the DRGs and DBGs and describe their basic structural 
properties. In this section we also quantify the degree to 
which the structure depend on wavelength and the degree 
to which this depends on J — if color. In Sec HI we fit the 
2D surface brightness distributions of the galaxies with 
Sersic profiles, and determine their best fitting Sersic n- 
parameters and effective radii rg. In Secl5] we compare 
the distributions of and n-parameters for DRGs and 
DBGs, and for star forming and quiescent galaxies. In 
Sec [6] we present 24/im MIPS imaging of the sample, and 
show how the 24/im flux correlates with the size of the 
galaxies. In SecH we use different diagnostics to search 
for the presence of AGN in the sample In Sec|8]we explore 
the dataset for correlations between size and properties 
of the stellar populations. We derive relations between 
mass and size, mass and surface mass density, mass and 
velocity dispersion, and size and surface brightness (the 
Kormendy relation), and compare to local relations to 
constrain a possible evolution. In Secl9]we compare the 
results to model predictions and speculate on an evo- 
lutionary link between the observed galaxies, and local 
galaxies. In SecllO]we discuss the uncertainties and pos- 
sible caveats of the analysis, and in SectTT]we summarize 
the results. 

We assume a standard cosmology (/iq = 0-7, = 
0.3, ^2 A = 0.7) throughout the paper, and all quoted mag- 
nitudes are in the Vega system. When deriving masses 
we assume a Salpeter initial mass function. 

2. DATA: SELECTION, PHOTOMETRY AND 
PHOTOMETRIC REDSHIFTS 

We used the FIRES data to construct a sample of 
galaxies with photometric redshifts z^i^ot ^ 2 in the 
MS 1054-03 field. P hotometry and photome t ric red shifts 
were adopted from iForster Schreiber et al.l (|2006l ) . We 
distinguish between distant red galaxies (DRG), selected 
by the criterion J — Ks > 2.3 which efficiently iso- 
lates rest-frame optically red galaxies at z > 2, and dis- 
tant blue galaxies (DBGs), those that do not meet the 
J — Ks > 2.3 criterion but have Zphot > 2. 



2.1. NICMOS Observations 

We obtained nine NICMOS pointings in the F160W 
band {Hpieow) with camera 3 (52'' x 52'', O'.'203/pix). 
The pointings were chosen to include as many DRGs as 
possible and as a secondary criteria, to include as many 
z > 2 DBGs as possible. Using this scheme we end up 
with a sample of 27 DRGs and 14 DBGs with NICMOS 
observations. At the time of observation, the NIC3 cam- 
era was out of focus, which result in a slightly broader 
PSF, than when the camera is in focus. In FigH] we 
show the layout of the NICMOS pointin gs on the FIRES 
K-band image (iForster Schreiber et al.l l2006). 

Each pointing was split up into six sub-pixel dithered 
exposures to allow for a better sampling of the point 
spread function and cosmic ray rejection. The six point- 
ings were reduced and combined using the following 
recipe: a first order estimate of the sky, a pedestal sig- 
nal (which is present in all NICMOS images) was sub- 
tracted from the individual frames using the IRAF task 
pedsky. Next, the background subtracted images were 
cross correlated to determine their shifts with respect to 
each other, using the iraf task crossdrizz. Using these 
shifts the background subtracted images were drizzled 
to a common coordinate system, with a finer pixel scale 
(with pixfrac= 0.65), using the iraf task drizzle. The 
drizzled images were then median combined and the me- 
dian combined image de-drizzled back to the pixel scale 
and coordinate system of the individual images, using 
the iraf task blot. The "blotted" images were then com- 
pared to the original images to create cosmic-ray maps, 
and cross-correlated to refine the shifts. Finally the cos- 
mic ray corrected, background subtracted images were 
drizzled using the refined shifts. The final combined im- 
ages have a pixel scale 0.1", and an exposure time of 
1.5 hours. The combined drizzled images showed con- 
siderable residual large scale (~ 25") features. These 
were successfully fitted and subtracted from the com- 
bined drizzled images, using background maps generated 
by SEXTRACTOR with options set to mask out objects 
and save background maps estimated using a fine mesh 
fitting. The masking thresholds passed to SEXTRACTOR 
was chosen very conservatively to make sure that size es- 
timates of faint galaxies and the outer parts of low surface 
brightness galaxies were not affected. 

2.2. ACS Observations 

The MS 1054 field was observed with ACS for 2 hours 
in the F606W-band {Vpeoew) and for 5 hours in each 
of the F775W (iF77^ w) and F850LP {zpsbOLp) bands 
(|Blakeslee et al.feoOGf ). Most of the DRGs in the sample 
are very faint in the rest-frame UV emission sampled by 
ACS, so we stacked the three ACS images to increase the 
signal to noise. The summed VF606iy + ^F775W^ + ^F850LP 
image has a total exposure time of 12 hours, and a pixel 
scale of 0.05". Details of the re d uction of the ACS data 
can be found in lBlakeslee et all (|2006l ). 

2.3. Spitzer IRAC Observations 

We observed the FIRES MS1054 field with the Mid- 
Infrared IRAC instrument on-board the Spitzer Space 
Telescope with a series of dithered observations result- 
ing in combined 5.5' x 5.5' images with exposure time of 
~3.2 hours in each of the 3.6, 4.5, 5.8 and 8.0 jiia bands. 
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Center: R,A. 10 56 58.15 Dec -03 36 59.7 
Layout of 9 NICMOS pointings on the FIRES K-band image (IForster Schreiber et aD[2006r ). North is up, east is to the left. 



The data reductio n and photometry w as c arried out in 
a similar way as in iLabbe et alJ dSPOG) and "Wuyts et alJ 
12007) and details of the data reduction and photometry 
method are described in Labbe et al (in prep). Here we 
briefly describe the data reduction and photometry. We 
started with the basic calibrated data (BCD) as provided 
by the Spitzer Science center. We applied a series of pro- 
cedures to reject cosmic rays and remove artifacts such 
as col umn pull-down, m uxbleed, and the "first frame ef- 
fect" (|Hora et al.l l2004l ). Finally, the frames were reg- 
istered to a 2 X 2 block averaged (0'.'2396 pixel scale) 
versi on of the existing ISAA C 5.5' x 5.3' K-band mo- 
saic ( Forster Schreiber et al.| [2006). Unfortunately, even 
though the size of the FIRES K-band mosaic is similar 
to the size of the deep IRAC pointings, the overlap is 
only ~ 70%, due to different centering and rotation of 
the two data sets. 

The main challenge in doing IRAC photometry is a 
proper treatment of source confusion. Information on 
the position and shape of the K-band detected objects 
were used to fit and subtract the fluxes of neighbor- 
ing sources. Each K-band source is isolated using the 
S Extractor "segmentation" map and convolved indi- 
vidually with a special kernal to match the IRAC PSF. 
Next all convolved sources were fitted to the IRAC im- 
age, leaving only the fluxes as free parameters. Sub- 
sequently the best fitting fluxes of neighboring sources 
were subtracted to remove contamination. This correc- 
tion is relatively small for most of the galaxies. The 
contribution to the total 3.6/im-band flux from neigh- 
boring sources is < 10% for 21 of the DRGs, 10 - 20% 
for 5 of the DRGs and - 40% for 1 DRG (with sim- 



ilar numbers in the other bands). We measure fluxes 
on the cleaned images, within 3'' apertures, and apply 
to each source an aperture correction to scale the fluxes 
by the "color" apertures defined in the FIRES catalog 
(the correction is the ratio of the K-band flux in the 
color aperture to the K-band flux within the 3'' aper- 
ture). The end product is a photometric catalog with 
consistent photometry from optical to MIR wavelengths 
with 12 filters {UBVVf606wIf814W JHK^IRAC), from 
which we extract the SEDs of the z > 2 galaxies in the 
sample. Note that the Vf606W and If814W filters in the 
catalog are from earlier WFPC2 observations, not the 
ACS observations discussed in Secl221 



2.4. Spitzer MIPS Observations 

The MS1054 field was observed with the Multi-band 
Imaging Photometer for the Spitzer Space Telescope 
(MIPS; iRieke et al.l \2004) as part of th e GTO Lens- 
ing Cluster Survey (e.g. lEgami et al.l|2007l ). The images 
were obtained using the 24/im channel (Ac = 23.7/im; 
AA = 4.7/im) which uses a 128x128 BIB Si: As array 
with a pixel scale of 2^'55 pixel"^. The data were re- 
duced and combined with the Data Analys is Tool (DAT) 
developed by the MIPS instrument team ([Gordon et al.l 
2005), and a few additional p rocessings; steps were applied 
as described in lEgami et al.l ([2006). The resulting mo- 
saic has a pixel size of 0'.'5. The total integration time 
is 3600 s over most of the 5^x5' field centered on the 
cluster, and is as much as 4800 s in a ~ 30''-wide strip 
crossing the cluster center because of the way multiple 
data sets were taken. The 5cr detection limit is ~50 jaJy 
for a point source. 
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We applied the same scripts to the mosaic to de- 
blend the light from confused sources and perform aper- 
ture photometry, as was applied to the IRAC data (see 
Sec J2.3p . We derive fluxes in ^" apertures after cleaning 
for flux from neighboring sources, and applied aperture 
corrections to derive total fluxes. In TabU] we list the 
24/im MIPS flux of the galaxies in the sample. A larger 
fraction of the DRGs than the DBGs are detected, both 
in the full redshift range (55% vs 15%), and at redshifts 
z < 2.5 (70% vs 25%). This difference is most likely 
caused by higher average star formation rate in the DRGs 
compared to in the DBGs (see Secl6]) 

2.5. SED Modeling 

We fitted the rest-frame UV-NIR SEDs of the galax- 
ies using the BC2QQ 3 stell ar population synthesis models 
(|Bruzual fc Chariot] 120031 ). assuming solar metallicity, a 
Salpeter IMF between 0.1 and 100 Mq and a Calzetti ex- 
tinction law. For the purpose of determining the stellar 
mass, dust content, age and star formation rate of the 
galaxies we fitted models with three different star forma- 
tion histories: a single burst model with no dust (SSP), 
an exponentially decaying star formation rate model, 
with a characteristic timescale of 300Myr (tau) and a 
constantly star forming model (CSF), both with variable 
amounts of dust. For each of the star formation histories, 
we constrained the time elapsed since the onset of star 
formation to > 50Myr, to avoid unrealistically young 
ages. 

To parameterize the star formation history of the 
galaxies, we divided the sample into "quiescent" and 
"star forming" galaxies, based on whether the SSP or 
CSF model provided a better fit to the SED. We chose 
to use only these two extreme star formation histories 
to limit the number of free parameters, and keep the 
division as simple as possible. Ideally one would make 
the division based on specific star formation rate (sSFR), 
but this quantity is not well constrained from SED fits. 
We emphasize that galaxies are generally expected to lie 
between these extremes. Our goal is to divide the popu- 
lation in two broad groups, based on their star formation 
rate, not to determine accurate star formation histories 
of individual galaxies. We note that the distinguishment 
between the SSP and CSF models is only used to divide 
the sample in to star forming and quiescent galaxies. For 
the purpose of deriving parameters of the SED fits, we 
also include the tau model, to avoid biases which could 
be introduced by fitting only the two extreme star for- 
mation histories. 

In table [1] we list properties of the DRG and DBG 
samples. 26%/15% of the DRGs/DBGs are classified as 
quiescent, and the remaining 74%/85% are classified as 
star forming. We note that DRG 25 is labeled star form- 
ing, but is equally well fit by the quiescent model (the 
of the CSF model fit is only marginally better than 
the of the SSP model fit). The derived stellar masses 
of the DRGs are in the range [1.8-55.4] x lO^^M©, with 
a median (M)^^^ = 16.9(±4.2) x lO^^M©. The masses 
of the DBG are in the range [0.4-14.0] x lO^^M©, with a 
median (M)^^^ = 2.5(±0.7) x lO^^M©, a factor of - 7 
lower than for DRGs. This difference in mass is in agree- 
ment with previous studies and is mainly a consequence 
of the DRGs having larger mass to light ratios (M/L) 
than the DBGs, due to older ages and a higher dust 



conte nt (see iForster Schreiber et al.l |2004 iLabbe et al.l 
[2QQ5I ). While de generacies between estimates of e.g. dust 
content and stellar age from restframe UV-NIR SEDs fits 
can b e substantial, stellar mass es are in general quite ro- 
bust ( van Dokkum et al] 12006^. and more precise than 
stellar mass estimates derived from rest frame K-band 
emission alone, due to the large observed var iations in the 
rest f rame M/Lk of galaxies at z :^ 2.5 (see lLabbe et al.l 
[2005h . 

Further general details on the method, ingredients 
of the SED modeli ng, and its strengths and lim ita- 
tions are des c ribed in lForster Schreiber et al] (|2004f ) and 
IWuvts et all(|2007[ ). A detailed analysis of the SED mod- 
eling for the present dataset will be presented in Forster- 
Schreiber et al (in prep.), but the main uncertainties and 
caveats related to the mass estimation are discussed in 
SecHni 

3. GALAXY STRUCTURE 

In Fig|2]we show a gallery of the DRGs in the sample 
as they appear in the NICMOS images and in the stacked 
ACS image. The DRGs show a range of structural prop- 
erties in the NICMOS images. About half of them are 
compact, the others are extended. At first glance, one 
might classify some of these galaxies as ellipticals, irregu- 
lars or mergers, but we abstain from such a classification 
here, and derive quantitative parameters, concentrating 
on half-light radius and Sersic index (see SecH]). Visual 
inspection of the NICMOS images suggest that about 
37% of the DRGs have close faint (DRG 4, 11, 22) or 
bright (DRG 13, 24) companions or assymetries (10, 14, 
17, 26, 27) which indicate they could be in the process of 
minor or major mergers. Unfortunately it is not possible 
to derive reliable photometric redshifts for any of the pos- 
sible companions due to their faint magnitudes and/or 
small angular seperations. We can not establish whether 
they are real physical associations. The difference be- 
tween the appearance of the DRGs at rest-frame optical 
and rest-frame UV wavelengths is striking: While most 
of the DRGs are detected at high S/N in the NICMOS 
images, more than half of them are only marginally de- 
tected, or not detected at all in the ACS images. Some of 
the most extreme examples are DRGs 3, 13 and 16 which 
are among the brightest galaxies in the NICMOS images, 
but are undetected in the ACS images. The DRGs that 
are detected in the ACS images in some cases have quite 
different structures in the NICMOS and ACS images. 
The most pronounced examples are the DRG 17, 23 and 
26, which are among the largest galaxies in the NIC- 
MOS images. In the NICMOS image DRG 23 appears 
like a nice symmetric edge on disk galaxy, but in the 
ACS image the disk is almost invisible, and the emission 
is dominated by a an off-center "blob" at the edge of 
the disk. In the NICMOS image, DRG 26 appears like a 
face-on disk galaxy dominated by a bright central bulge, 
and a large lower surface brightness diffuse disk with a 
few off center blobs of emission, but in the ACS image 
the galaxy appears like a "chain" galaxy, dominated by 
a blob of emission in one end of the chain rather than 
by emission from the central bulge. This galaxy is de- 
tected in relatively shallow Chandra X-ray observations 
and thus hosts an AGN (see SeclTj). In the NICMOS im- 
age, DRG 17 appears to have a bright central bulge, and 
two asymmetrical arms, which may be either spiral arms 
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Fig. 2. — Images of the 27 DRGs in the sample. NICMOS ifi?i6ow-band (1.5h) and stacked Vfqqqw + iprrbw + zpsbOLP (12h) ACS 
images are shown side by side. The 21 galaxies in the first three columns have exponential disk-like surface brightness profiles, while the 6 
galaxies in the fourth column have "de Vaucouleurs-like" surface brightness profiles. Four galaxies (DRG 5, 20, 21 and 24) are better fitted 
by exponential disk-like surface brightness profiles, but are formally unresolved. The size of the images is 5^^ X 
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Fig. 3. — Images of the 14 DBGs in the sample. NICMOS ifi?i6ov^^-band (1.5h) and stacked Vfgogw + 'i'F775W + ^F850LP (12h) ACS 
images are shown side by side. The 12 galaxies in the first three columns have exponential disk-like surface brightness profiles, while the 2 
galaxies in the fourth column are better fitted by "de Vaucouleurs-like" surface brightness profiles (but are formally unresolved). The size 
of the images is 5" x 5" . 
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or tidal tails induced by a recent merger. In the ACS im- 
age all that is detected is a double central point-source, 
and a bit of diffuse emission. This galaxy is has been 
directly detected with SCUBA , indicating; that i t hosts 
very intense star formation (Knudsen et al. "2005'). 

In Figl3]we show a similar gallery of the DBGs in the 
sample. Comparing this figure to FiglSl the main differ- 
ences are 1) that a larger fraction of DBGs (57%) have 
faint (DBGs 3, 5) or bright (DBG 10, 11) nearby com- 
panions or show assymetries (DBGs 2, 4, 6, 8, 9, 13) 
that may be contributed to minor or major recent or 
ongoing merging (note that companion of DBG 7 has a 
lower z^hot^ and that DBG 10 and 11 which are two com- 
ponents of the same major merger are counted as two 
seperate galaxies), 2) all the DBGs are detected in the 
ACS image, and 3) their structure (with the exception 
of DBG5) are similar in the NICMOS and ACS images. 

To illustrate the last point, we calculated the distance 
between the luminosity weighted central pixel position of 
the light distribution in the ACS and NICMOS images, a 
measure of the difference between the ACS and NICMOS 
structure. In FiglH this quantity is shown to correlate 
with J-K color. The median shift for the DRGs is larger 
(by a factor of ~ 3) than for the DBGs. According to 
the Mann- Whitney test there is a 99.6% chance of the 
mean shifts being different. We did not find evidence for 
a correlation between Ar and Rpimw band flux. Note 
that the fact that the galaxies are very faint in the very 
deep ACS imaging informs us about their colors, but 
not about potential morphological differences. In fact, it 
makes quantitative statements very hard. Hence, we use 
only a very simple indicator, the centroid difference, to 
study the difference between rest-frame UV and optical. 
A proper analysis would comp are sizes, asymmetry, etc. 
(see e.g.. iPapovich et ani2005l ). 

The relatively relaxed centrally symmetric NICMOS 
structure and faint and more disturbed ACS structure 
of the DRGs are consistent with composite stellar pop- 
ulations, where relatively dusty, evolved red populations 
dominate the rest-frame optical structure (and stellar 
masses), and off-center blobs of ongoing star- formation in 
some cases dominate the rest-frame UV structure. This 
demonstrates that high resolution rest-frame optical ob- 
servations are essential for deriving structural informa- 
tion and indirectly, information on the dynamical state 
of massive red galaxies at 2: > 2 (e.g. sizes, Hubble types 
etc). The DBGs, on the other hand, have very simi- 
lar rest-frame optical and UV structure, suggesting that 
they are dominated by uniformly young populations of 
stars with little dust. This suggests that the conclusion 
about the size distribution and structural properties of 
LBGs b ased on the i r observed optical p roperties (e.g. 
[Ferguson et al.|[2QQ3 : iBouwens et ani2004l ) are probably 
"correct" since their rest-frame UV structure is likely to 
be representative of their rest-frame optical structure. 

Note that the prominent differences between the rest- 
frame UV and optical structure of the red galaxies in 
some cases may introduce systematic errors in photo- 
metric studies, based on aperture photometry from lower 
spatial resolution data, as flux in bluer waveband aper- 
tures may originate in different physical regions of the 
galaxies than the flux in the redder waveband apertures, 
making the assumption that a monotonic star formation 
history can be fitted to the integrated flux of the galaxy 




Fig. 4. — Offset Ar between the luminosity weighted cen- 
tral pixel position of the light distribution in the ACS and 
NICMOS images, as a function of their J — Ks color. The 
luminosity weighed central pixel is calculated as (xc,yc) = 

(T.x,y ^ • ^(^' y)/ T.x,y ^(^' 2/)' T.x,y V ' ^(^' V)) / T.x,y ^(^' 2/), whcrC 

i{x^y) is the intensity in the pixel with coordinates (x,y), and x 
and y run over all pixel values with intensities > 3cr in a 3'' x 3" 
box centered on the galaxy . The red symbols represent DRGs, 
the blue symbols represent DBGs. Large symbols are detected at 
high significance in the ACS images, small symbols are marginally 
detected. The dashed red line is the median offset of the DRGs 
(Ar}^^(-, = 0.24 ± 0.03 arcsec, the dotted blue line is the median 
offset of the DBGs (Ar}^^^^ = 0.08 ± 0.03 arcsec . 

invalid. 

4. PROFILE FITS AND SIZES 

We fitted the 2D surface brightness distribution of the 
galaxies in the sample, using galfit (|Peng et al.ll2QQ2l ) 
and a Sersic profile: I{r) = l{0)exp [&n(^/'^e)^^^] ^ which 
has been shown to be a good representation of the sur- 
face brightness of a range of galaxy types from ellipticals 
(n=4) to disks (n=l). The effective radius rg, is enclosing 
half the light of the model and bn is a normalization con- 
stant. GALFIT returns the effective radius along the semi- 
major axis of the model, a^. This can be converted to 
the "circularized" effective radius rg = agVl — where e 
is the projected ellipticity of the galaxy model. To limit 
the degree of freedom in the fits, we fixed n at the val- 
ues 1, 2, 3 and 4 and identified the best fitting model by 
comparing the of the four fits. The best fitting values 
of n and the corresponding best fitting values of rg are 
listed in Table El 

Galfit fits models which are convolved with the PSF 
(estimated directly from stars in the field) to the data, 
and is thus in principle capable of measuring sizes smaller 
than the size of the PSF. There are four stars in our NIC- 
MOS pointings which are sufficiently isolated and bright 
to be used as PSF stars. We repeated the analysis us- 
ing each star in turn to estimate the effect of the choice 
of PSF star on the distribution of the derived sizes and 
radial shape parameters. No systematic dependence on 
the choice of PSF was found. The la mean uncertainty 
of the derived size of a given galaxy due to the choice of 
PSF is ~ 14%. The derived n-parameters are not signif- 
icantly affected by the choice of PSF. We also estimated 
the effect of the additional background subtraction (de- 
scribed in Sec. 12. ip . and the size of the fitting region on 
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Fig. 5. — Comparison of sizes derived from the NICMOS 
Hpimw and the FIRES ISAAC H band images. The small sym- 
bols represent galaxies which are unresolved in the NICMOS data 
(plotted at the NICMOS resolution limit). The dotted line rep- 
resent the resolution limit of the ISAAC data. The numbers cor- 
responds to the IDs of the DRGs. To illustrate the limitation of 
the ISAAC data for measuring the smallest sizes we plot galax- 
ies at their measured ISAAC sizes, rather than correcting these to 
the resolution limit of the ISAAC data. There is a relatively good 
agreement at sizes re > 0.2 arcsec where the galaxies are resolved 
in the ISAAC images, at smaller sizes, there is a large scatter. 
The three galaxies that are significantly larger in the NICMOS im- 
ages (DRG 6, 17 and 27) have low surface brightness components 
that are not detected by ISAAC. The inset compares the sizes de- 
ri ved here from the NICMOS data (rejv/c) with the sizes derived 
in ITruiillo eTaT. (2006) from the FIRES data (rcToe), for the 14 
galaxies which are in both data sets. 

the derived sizes. No systematics offsets are found for 
these and the mean uncertainty due to these effects are 
of the order of 1%. Added in quadrature the total error 
due to additional background subtraction, and choice of 
PSF and the size of the fitting region is less than 15%. 
We estimated the minimum reliable size derived by gal- 
fit by applying the size fitting analysis to the four PSF 
stars, using the other three stars as PSF models. The 
mean size of "bona fide" unresolved objects is in this 
way estimated to be {re) stars ~ 0'-'030 ± 0^015. Galaxies 
with derived sizes larger than 0'!075 are thus resolved at 
the 3cr level in the NICMOS data. Four of the DRGs 
and 2 of the DBGs are unresolved in the NICMOS im- 
ages, corresponding to a fraction of about 15% for both 
samples. In the following, we adopt 0'!075 as an upper 
limit for the size of the unresolved objects. We note that 
the galaxies that are unresolved in the NICMOS images 
have ACS sizes consistent with the upper limits derived 
from the NICMOS images. 

As an independent check we fitted the surface 
brightness distributions and derived sizes for the 27 
DRGs in th e deep FIRES IS AAC H-band observation 
([Forster Schreiber et al.l 12004"). This data is deeper but 
has a courser resolution (FWHM^ 0.5''). In Fig (5] we 
compare the sizes derived for the DRGs in the ground 
based and space based data. There is no significant 
systematic offset between the sizes measured in the two 
data sets (median {reisAAC /^^nicmos) = 1.06 ±0.17), 
but there is a relatively large scatter, especially at sizes 
re-isaac ^ 0.2'' where the galaxies are unresolved in the 
ISAAC data. Three galaxies which are resolved in both 
data sets, have significantly larger sizes in the NICMOS 
data, than in the ISAAC data. Visual inspection of the 
NICMOS images (see Figl2j) reveals that this is proba- 
bly due to low surface brightness components which are 



not properly detected in the ISAAC data. A subset (14) 
of the DRGs s tudied here are also part of the sample of 
ITruiillo et all (|2006[ ). who derived sizes from the FIRES 
data in a very similar way as is done here for the ISAAC 
data. In the inset of Fig (5] we compare the sizes of these 
galaxies derived by Trujillo et al, (2006) from the ISAAC 
data with the sizes derived here from the NICMOS ob- 
servations. Again, a relatively large scatter is evident, 
especially at sizes re < 0.2", where the sizes determined 
from the ground based data is uncertain by at least a 
factor of 2. 

In addition to the systematic uncertainties in determin- 
ing the angular sizes of the galaxies, there are uncertain- 
ties in the conversion of the angular sizes into physical 
sizes, due to uncertainties in the photometric redshifts. 
These are discussed in SeclTOl 

5. SIZE AND N-DISTRIBUTIONS 

In Tab|2]we list the best fitting n and rg parameters. 
The galaxies show a broad size distribution, ranging from 
compact unresolved galaxies with Ve < Ikpc, to large 
galaxies with rg > lOkpc. Most of the galaxies are best 
fitted by exponential disks. In the following we compare 
the size and n-distributions of different subsamples. 

In Figiniwe plot the n-distribution of DRGs and DBGs. 
Both samples are dominated by galaxies with exponential 
disk profiles: 80% of the resolved DRGs are best fitted by 
"exponential disk-like" surface brightness profiles (n = 
1, 2) while 20% are better fitted by "de Vaucouleurs-like" 
surface brightness profiles. All the resolved DBGs are 
best fitted by exponential disk-like laws. The KS-test 
yields a 21% chance that the n- values were drawn from 
the same parent distribution. The distribution of radial 
shape parameters of the DRGs are similar to what was 
found in the UDF where 5 out of 6 DR Gs wh ere found to 
be best fit by exponential disk profiles (|Toft et al. 2005). 

The Te distribution of the DBGs is similar to that of 
the DRGs. The KS-test yields a 64% chance that the 
Te values were drawn from the same parent distribution. 
In FiglTlwe take into account that the DBGs are on av- 
erage not as massive as the DRGs by normalizing the 
sizes by the l ocal mass-size rela tionship of local galaxies 
in the SDSS (<Shen et al.l l2003f ). Note that we through- 
out the paper take into account the difference between 
the Salpeter IMF assumed here and the Kroupa IMF 
assumed by IShen et al. {206§) by divi ding ou r derived 
masses by two before comparing t o IShen et al.i (2003). 
As a possible caveat we note that .Shen et al.l (|2003 ) de- 
rive masses in a different than done here. The mass- 
normalized comparison further supports the similarity of 
the size distributions. The KS-test yields a 50% chance 
that the mass-normalized rg values were drawn from the 
same parent distribution. Furthermore it can be seen 
that the sizes of the galaxies in the sample are on av- 
erage smaller by a factor of about three than late type 
galaxies of similar mass in the local universe (see Sec J8.ll 
for a more details). Here, and in the following section we 
normalize all sizes u sing the local ma ss-size relation of 
late- type galaxies in I Shen et al.l (|2003l ). as the vast ma- 
jority of the galaxies in the sample are best fitted by ex- 
ponential disk surface brightness profiles. Alternatively 
we could have normalized the sizes of galaxies with de 
Vaucouleurs like surface brightness profiles, by the local 
relation for early- type galaxies, however as the local early 
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FlG. 6. — Left: Distribution of Sersic index for resolved DRGs 
(shaded) and DBGs (hatched). Right: Distribution of Sersic index 
for resolved quiescent (shaded) and star forming (hatched) galaxies 
(DRGs+DBGs). In the left plot 80% of the DRGs are better fitted 
by exponential disks than r^l^ law surface brightness profiles. In 
the right plot 83% of the star forming galaxies are better fitted by 
exponential disks than r^l^ law surface brightness profiles 
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Fig. 7. — Left: Distribution of derived circularized effective radii 
for DRGs (shaded) and DBGs (hatched). Right: Same as left plot, 
but normalized by the local mass-size relationship for late-type 
galaxies in SDSS (Shen et al. 2003): Each size has been divided 
by the size of a late type galaxy of similar mass in SDSS. In both 
plots, the distributions are not significantly different. 

and late type relations are very similar in the mass range 
of the "de Vaucouleurs" law galaxies in the sample (see 
Fig Ho]), it would not significantly change the normalized 
sizes. 

While the sizes of the galaxies do not seem to depend 
on their J-K color, there is a strong correlation with star 
formation activity. In Fig. [8] we compare the size dis- 
tribution of star forming and quiescent galaxies. There 
is a clear trend that quiescent galaxies are smaller than 
star forming galaxies. Four of the quiescent DRGs (57%) 
are unresolved in the NICMOS images, while none of the 
star forming DRGs are unresolved. As shown in FiglH] 
(right) the size difference is even more pronounced when 
normalized by the local mass-size relation. From this 
figure it can be seen that the quiescent galaxies are on 
average smaller by a factor of about 5 than late type 
galaxies in the local universe, while star forming galaxies 
on average are smaller by a factor of about 2. The mass 
distribution of the star forming and quiescent DRGs are 
similar (the KS test yields a 80% chance of them being 
drawn from the same parent mass distribution) It will 
be interresting to determine whether star forming galax- 
ies have lower mass densities or are dense systems with 
widely distributed star formation. In FiglH we compare 
the n-distributions of the resolved star forming and qui- 
escent galaxies. All of the resolved quiescent galaxies and 
83% of the resolved star forming galaxies are better fit- 
ted by exponential disk than de Vaucouleurs like surface 
brightness profiles. The KS test yields a 54% chance of 
the n- values of quiescent and star forming galaxies being 
drawn from the same parent distribution. 

6. CORRELATIONS BETWEEN SIZE AND MIPS 24)Lim 
EMISSION 
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Fig. 8. — Left: Distribution of derived circularized effective 
radii for quiescent (shaded) and star forming DRGs (hatched) his- 
tograms. Right: Same as left plot, but normalized by th e local 
mass-size relationship for late-type galaxies in SDSS (S hen et al.l 
120031 ') . The quiescent galaxies are significantly smaller than the 
star forming galaxies. 

At 1.5 < ^ < 2.5 the 24/im MIPS filter samples rest- 
frame ~ 6 — 10/im, and the 24/im flux offers a powerful 
method for differentiating between galaxies with SEDs 
dominated by dusty star formation and/or AGN, both of 
which produce substantial MIR emission, and quiescent 
galaxies whose stellar flux continues to drop longwards of 
~ 2/im (e.g. Webb et al. 2006). In pure starburst galax- 
ies the MIR emission is dominated by PAH features, 
that are stron g relative to t he underlying dust contin- 
uum (e.g. Smit h et"al] l2007l ). The hard radiation field 
of an AGN destroys PAH carriers, and the continuum 
emission from hot small dust grains is strong throughou t 
the MIR (jGenzel h CesarskvllSOOO : iLaurent et al.ll2000[ ). 
At z > 2.5 the sensitivity to star formation is reduced 
drastically as the PAH features are redshifted out of the 
24/im band, so in the following we will concentrate on 
the galaxies in our sample with z < 2.5, which leaves 
19 DRGs and 4 DBGs. Assuming that the 24/im flux is 
dominated by star formation, the PAH and MIR emis- 
sion can be used to estimate the cu rrent star formation 
rate of the galaxies fWu et alj l2005[ ). Using an Arp 220 
SED template, we extrapolated from 24/im/(l -h z) flux 
to rest-frame 6.75/im flux. We then estimated the total 
(8 — 1000/im) infrared luminosity Ljr through the ob- 
served 6.75/im— Fp/i^ relation calibrated locally with the 
Infrared Space Observatory (|Elbaz et al.l l2002f ) , which 
shows a factor of 2 scatter. This value can then be con- 
vert e d into a SFR using the Ljr — SFR relationship of 
iBelll (|2003f ). with an expected scatter of at least a fac- 
tor of two. Additional systematic uncertainties are in- 
troduced in the first step extrapolation. Adopting a M- 
82 like template instead of the Arp 220 te mpl ate leads 
to a factor of two lower SFR ( Webb et al .112 006f) . while 
adopting the model templates of iDale fc Heloul ([2QQ2h 
can result in factors of 2-6 lower SFRs (|Papovich et al.l 
i2007l) . Uncertainties in the photometric redshifts trans- 
lates into a factor of 2.5 uncertainty in the derived SFR 
(Papovich et alj |2006[ ). Added in quadrature the ex- 
pected uncertainties in 24/im-SFR conversion amount to 
about a factor of ^ 7. Furthermore, the SFRs derived in 
this ways are upper limits, as AGN may contribute to, or 
in some cases dominate the 24/im flux, leading to over- 
estimated SFRs. In Fig|9] (left) we plot the specific star 
formation rate derived from 24/im flux versus the ratio of 
flux in the MIPS 24/im and IRAC 4.5/im bands. There is 
a clear correlation, galaxies with higher F24/im /^4.5/im 
ratios have higher derived sSFRs. This ratio is a purely 
observational quantity which does not depend on extrap- 
olations and (only weakly, through the assumed photo-z) 
on assumptions about the SED, and thus is a good simple 
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Fig. 9. — Ratio of flux in the MIPS 24/im and IRAC 4.5/im bands, versus the sSFR derived from the 24/im flux (left) and normahzed size 
(right) for galaxies in the sample with 2 < ^p^ot < 2.5. Error bars are one sigma and include only photon and confusion noise associated 
with the 24/im photometry, not systematic errors related to the conversion from flux to SFR. For clarity errors on the sSFR are only 
shown in the left plot, and errors on F24/im/-?^4.5/im are only shown in the right plot. Galaxies with negative measured fluxes are shown at 
Icr upper limits (indicated by arrows). There is a relation between the derived speciflc star formation rate and the -F24/im/-?^4.5/im ratio, 
galaxies with larger ratios have larger sSFRs. There is also a relation between F24/im/-?^4.5/im ratio and size, galaxies with larger ratios on 
average have larger sizes. 



estimator of star formation/ AGN activity in the galax- 
ies. The origin of the relation is the fact that F24^m 
roughly scales with the SFR (and redshift) of the galax- 
ies and F4.5^m (which samples the restframe J-band at 
z ~ 2.5) correlates linearly with their stellar mass (with 
a considerable scatter). 

Four of the five galaxies in the relevant redshift range 
(with IRAC coverage) which were classified as quies- 
cent based on their optical-IRAC SEDs have very low 
inferred sSFR24jj,m (consistent within la with no on- 
going star formation) and F24/im/^4.5/im < 1 (within 
Icr), while one (DRG 5) formally has a higher derived 
sSFR and F24/im/^4.5/im, but is only a ~ 2cr detection 
at 24/im. Twelve of the thirteen galaxies (with IRAC 
coverage) which were classified as star forming have high 

.5/im > 1, while 

one (DRG 25) is undetected at 24/im. Note that this 
galaxy which is almost equally well fit by the star form- 
ing and quiescent SED model (see Sec j2.5[ ) Four galaxies 
in the relevant redshift range did not have IRAC cov- 
erage and are therefore not plotted in FiglH but we 
note that the two quiescent galaxies (DRG 20, DBG 
3) has sSFR24jj,m consistent within la with no ongo- 
ing star formation, while the two star forming galaxies 
(DRG 19, DBG 13) have sSFR2Aiim = 2.4, 11.4 respec- 
tively. We did not find evidence for a dependency of 
the derived sSFR24ij,m on stellar mass in our sample. 
We note that the sSFRs for these galaxies are much 
higher than those of galaxies of similar mass at lower 
redshift. This suggests that a significant fraction of mas- 
sive galaxies was still forming stars at z ^ 2.5 whereas 
they are quiescent today (see also I van Dokkum et aH 
I2QQ4I : iPapovich et"aD 120061 ). In FigM (right) we plot the 
^24/im/^4.5/im ratio as a function of normalized size. 



Galaxies with low F24/im/^4.5/im ratios are significantly 
smaller than galaxies with large F24/im/^4.5/im ratios. 
This provides independent evidence for the relation be- 
tween star formation activity (derived from SED fits) and 
size presented in Sec 15] While it is hard to disentangle 
the contribution of star formation and AGN to the mea- 
sured 24/im flux, the MIPS observations independently 
confirm that the galaxies classified as quiescent do not 
host significant ongoing star formation or AGN activity, 
and the interpretation that they are small and red due 
to very compact old stellar populations, rather than dust 
enshrouded central AGN. In SecH we explore the pres- 
ence of AGN in the fuh sample. Note that 12/13 DRGs 
at 2 < z < 2.5 with MIPS 24/im detections have inferred 
total infrared luminosities comparable to local Ultra Lu- 
minous Infrared Galaxies (ULIRGS, Lm > lO^^L©) 
which place them among the most powerful starbursts 
known, in some cases with SFR > lOOOM©. This is 
consistent with the fact that two of the brightest DRGs 
at 24/im (DRG 13 and 17) are also di rectly detected 
in JC MT-I-SCUBA 850/im imaging (see iKnudsen etlH 
l2005l ). S imilar high total infra red luminosities are de- 
rived by Papo vich et al.l (|2006l ) for DRGs in the same 
redshift range. 

7. AGN 

Previous observations of DRGs suggest that some of 
them host AGN. Different techniques for detecting the 
presence of AGN find different fractions, mainly due 
to different selection bi ases, and sensitivit i es. B ased 
on optical spectroscopy Ivan Dokkum etlol (|2003r ) find 
that 2 out of 6 DRGs with emission lines are consis- 
tent with AGN activity, c orrespo nding to 33%. Based 
on NIR spectroscopy, K riek et all ([2006b) find 4 out 20 
K-selected galaxies between 2 < 2; < 2.7 (20%) with se- 
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cure NIR spectroscopic redshifts host AGN. Based on 
relatively shallow X-ray observations (91 ks exposure 
with the Chandra X-ra y telescope) of the MS 1054-03 
field, iRubin et all (|2004f ) find that two out of 40 DRGs 
have x-ray point source counterparts, corresponding to a 
AGN fraction of 5%. Combining X-ray observatio ns and 
the shape of the MIR continuum, iPapovich et alJ (j2006) 
found an AGN fraction of 25% among a sample of 153 
DRGs in the redshift range 1.5 < z < 3.0. 

Here we estimate the AGN fraction in our sample 
and the infiu ence on our result s, by combining the X- 
ray imaging (|Rubin et al.ll200j ). with the shape of the 
MIR continuum, and visual inspection of the SEDs (for 
the presence of excess restframe UV fiux) and structure 
(for the presence of blue central point sources). Two of 
the DRGs in MS1054-03 field (DRGs 26 and 27) are de- 
tected as X-ray point sources, demonstrating that they 
host AGN, but in addition to these there may be other 
AGN which are too faint to be detected directly. None 
of the DBGs are detected in the rather shallow X-ray 
observations. 

Another way to locate AGN is by inspecting the IRAC 
colors of the galaxies. AGN stand out in t he 5.8 — 8.0 



4.5/im color-color plot (see fig.l in 
Adopting the empirical criteria of 



Stern et al 



Stern et al 



vs 3.6 
[20051) 

(j2005l). 5 of the DRGs in our sample have IRAC col- 
ors consistent with hosting AGN (DRGs 1, 6, 9, 21, 26), 
while two fall just outside the selection criteria (DRGs 
10 and 27). Another indicator of AGN activity is when 
galaxies with z > 2.5 (beyond which the sensitivity to 
starburst induced emission drops drastically) have sig- 
nificant MIPS detections. Three DRGs (8, 10, 26) fah in 
this category. 

In Tabjl] it is indicated which galaxies in the sample 
meets the different AGN selection criteria. There is a 
large overlap between the subsamples selected from X- 
rays, IRAC and MIPS. Both X-ray selected candidates, 
and 2/3 MIPS selected candidates are included by the 
IRAC color selection criterion. The MIPS and X-ray 
data are only deep enough to detect the brightest AGN, 
so the three galaxies which are selected from IRAC, but 
not the other techniques could either host fainter AGN or 
not host AGN at all (some contamination (< 20%) is ex- 
pected for the IRAC color selection method, IStern et al.l 
[20051 

Two DBGs have IRAC colors consistent with hosting 
AGN (DBGs 10 and 12). DBG 10 is potentially an on- 
going merger of two galaxies (with consistent photomet- 
ric redshifts, see Secl3j), one of which has a blue point 
source in the center, which is likely an AGN. DBG 12 is 
a blue, very compact galaxy which is unresolved in the 
NICMOS image, but resolved in the ACS image. Finally, 
one z > 2.5 DBG is detected by MIPS (DBG 5). This 
galaxy is very blue in the optical, suggesting it could host 
an AGN. 

The estimated AGN fraction in the DRG sample is thus 
between 7 — 30%, depending on which selection method 
is used, and 15% in the DBG sample. 

It is not straightforward to estimate the relative con- 
tribution of the possible AGN to the SED, but the most 
important point for the present analysis, is that 7 of the 8 
galaxies which are candidates for hosting AGN are well 
resolved in the rest-frame optical, (with sizes spanning 
the full range 1-8 kpc), and are classified as star form- 



ing, based on their SEDs, indicating that the possible 
AGN does not dominate the rest-frame optical emission 
and structure. Inspection of the SEDs of the AGN candi- 
dates confirms that the rest-frame optical emission is well 
fitted by stellar population models (with significant spec- 
tral breaks), but in all cases there is some evidence for 
excess rest-frame UV emission with respect to the mod- 
els, which could be attributed to AGN. The UV excess is 
however only significant for DRG 26 and 27 (the X-ray 
sources). Of the 8 AGN candidates, only one (DRG 1) 
has a central point-like sources in the stacked ACS im- 
age, suggesting that the possible AGNs in the remaining 
7 galaxies are obscured by dust. Visual inspection of the 
SED of DRG 21, (the only AGN candidate that is un- 
resolved in the NICMOS images), shows that it is well 
fitted by the SSP model with a strong 4000A break, but 
with excess fiux in Sjam band. Note that this galaxy is 
neither detected in X-rays or at 24/im. While the derived 
star formation rates, ages and masses, dust content etc., 
of the galaxies in question could be affected by the pos- 
sible presence of AGN, it is not expected to significantly 
affect the conclusions of this study, as the main corre- 
lation between size and star formation activity persists 
even when the AGN candidates are excluded from the 
sample. Furthermore, since the AGNs do not dominate 
the restframe optical emission of the galaxies, their influ- 
ence on the derived SED fit parameters is likely modest. 

8. THE RELATION BETWEEN STRUCTURE AND 
STELLAR POPULATIONS 

In this section we explore the data for correlations be- 
tween structural properties of the galaxies, and proper- 
ties of their stellar populations derived from the SED 
fits. 

8.1. Mass - Size Relation 

In Fig{TO] (left) the size-mass relation of the individual 
subgroups of galaxies are compared to each other and to 
the relations in the local universe (SDSS). It is evident 
that most of the galaxies are smaller than galaxies of 
similar mass in the local universe. It is not possible to 
determine whether the mass-size relation of the galaxies 
resembles the local distribution of late type or early type 
galaxies better, but since the majority of the galaxies are 
better fitted by exponential disk profiles, we will compare 
to the mass-size relationship of late-type galaxies in the 
following (unless noted otherwise). In FigfTOl (right) the 
mass-size relation is normalized by the local mass-size 
relation of late type galaxies. 

The median sizes of both DRGs and DBGs are smaller 
by about a factor of 2.5 compared to late type galaxies 
of similar mass in the local universe {{re/resDSs) drg ~ 
0.37 ± 0.08, {re/resDSs)DBG = 0-41 ± 0.21) 

A larger difference is found when comparing the me- 
dian sizes of quiescent and star forming galaxies: the 
median size of the quiescent galaxies is a factor of ~ 5 
smaller {{re/resDSs) = 0.19±0.03) than late type galax- 
ies of similar mass in the local universe, while the median 
size of star forming galaxies are only smaller by a factor 
of - 2 {{re/resDSs) = 0.45 ± 0.15). Note that if we 
normalize the sizes of the quiescent galaxies using the 
local mass-size relation of early- type galaxies, the me- 
dian size {{re/resDSs) = 0.23 ± 0.04 is still consistent 
with being a factor of 5 smaller than local early type 
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Fig. 10. — Left: Sizes (effective radii) as a function of stellar mass. Red symbols are DRGs, blue symbols are DBGs. "Flat" symbols have 
"exponential disk like" like surface brightness profiles, "Elliptical" symbols have "de Vaucouleurs-like" surface brightness profiles. Small 
symbols are unresolved in the NICMOS images, and are represented by an upper limit on their size. Open circles indicate galaxies with 
SEDs consistent with being quiescent, the rest have SEDs consistent with being star forming. Stars indicate (3cr) MIPS 24/im detections. 
Black dots indicate that the galaxy may host an obscured AGN (see Sec|7}. The curves are the mass-size relations of early- type (red) and 
late-type (blue) galaxies in SDSS (Shen et al. 2003). Right: Sizes, divided by the mass-size relation for late type galaxies in the SDSS (the 
blue curve in the left plot). Quiescent galaxies are on average a factor of 5 smaller than late type galaxies of similar mass in the SDSS, 
while star forming galaxies are smaller by a factor of 2. 



galaxies of similar mass. The star forming galaxies ac- 
tually divide into two distinct subgroups: the majority 
have {{re/resDSs) = 0.52 ± 0.14), but 8 galaxies have 
(re/resDSs) = 1—3, and are thus larger than late type 
galaxies of similar mass in the local universe. These 8 
galaxies all have disturbed structure in the NICMOS im- 
ages (see FigEl DRG- 13 17, 19, 26, 27 and FiglSl DBG- 
5, 10, 11), and ah are detected by MIPS. 

The observed evolution of the mass-size relationship 
is consistent with the evolution of the mass-size relation 
derived from the FIRES survey for galaxies at z ~ 2.5 
which were found to be a factor of 2 zb 0.5 smaller than 
galaxies of similar mass in the SPSS (based on size s de- 
rived from ground based data, iTrujillo et al.ll2QQ6l ). In- 
terestingly, from a study of the mass-size relation in the 
HDFN (|Papovich et al.ll2QQ5[ ) find that galaxies at - 2.3 
are "only" 40% smaller than galaxies at z ~ 1, so the 
majority of the size evolution must take place between 
< ^ < 1. 

8.2. Surface Mass Density 

In this section we combine the derived stellar masses 
and sizes of the galaxies in the sample, to derive surface 
mass densities. If we assume that mass follows light, 
the surface mass density of a galaxy within the effective 
radius is 

5^50 - (1) 

In FigUl] (left) we plot the derived surface mass density 
versus the total stellar mass, and compare to the mass 
densities derived for galaxies in the local u niverse in^ 
similar way from the mass-size relations of Shen et al. 
([2003 ) . The galaxies follow a similar relation as for late- 
type galaxies in the local universe (with a large scatter). 



with more massive galaxies being denser, but shifted 
to higher mass densities for a given stellar mass. In 
FigHH (right) we normalize the mass densities by the 
local late-type relation. In this plot it can be seen 
that there is no evidence for DRGs and DBGs follow- 
ing different M-S relations. The DRGs are more massive 
than the DBGs, but have similar sizes, which translates 
into higher surface mass densities. The median surface 
mass densities are {^50 /^50,s ds s) ^^jiq = 9.6 ± 4.2 and 
{^3o/^50,sdss)dbg ^ ^'^ ^ times higher than for 
late type galaxies of similar mass in SDSS for DRGs and 
DBGs respectively. The quiescent galaxies separate out 
from the star forming galaxies in this plot with much 
higher surface mass densities. The median surface mass 
density of quiescent galaxies is 1-2 orders of magnitude 
higher than in late type-galaxies in the local universe, 
with a median value of {^5o/^5o,SDSs)q = 36.0 ± 5.8 
while the median surface mass density of the star form- 
ing galaxies is {^5o/^50,sdss) ^ = 6-2 ± 3.2. The star 
forming galaxies separate into two groups, the majority 
has surface mass densities around the median value, but 
8 galaxies have surface mass densities which are lower 
than in late-type galaxies of similar mass in the local 
universe. These are the same galaxies which stood out 
in the mass-size plot (see Sec J8.ip . Note that if we nor- 
malize the sizes of the quiescent galaxies by the mass-size 
relation of local early type galaxies, the derived median 
{^5o/^50,SDSs)q = 33.4 ± 9.9 is still much higher than 
in the local early type galaxies of similar mass. 

8.3. Velocity Dispersions 

Here we assume that the galaxies have the same dy- 
namical structure as local early type galaxies and are 
part of the same homologous family as local galaxies. 
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Fig. 11. — Left: Average surface mass density within re as a function of stell ar mass. The l ines are the local relations for late (blue) and 
early (red) type galaxies in SDSS calculated from the mass-size relations of She n et all (|2003l ) . Right: Surface mass density, normalized by 
the surface mass density of late- type galaxies in SDSS (the blue line in the left plot). The star forming/quiescent galaxies are on average 
6/36 times denser than local late type galaxies of similar mass. Symbols are as in Fig jlOl 



hence their velocity dispersion can be estimated from 
their size and mass: 

log{M^) = 2log{(Jv) + log{re) + 6.07, (2) 

(|Jorgensen et al.l 1 19961 : Ivan Dokkum k Stanford! [2003 h. 
The assum ption of homology has been tested only out 
to z ^ 0.5 ( Ivan der Marel fc van Dokkumll2QQ6h . but the 
work of Ivan der Wei et al.l (|2005l ) suggest that these re- 
lations hold at least out to z ~ 1.2. However, as we shall 
see later it may not hold for the galaxies in our sample, 
as they will likely undergo significant st ructural evolu- 
tion from z ~ 2.5 to the present (see also I Almeida et all 

[2007h. 

'B ernardi et al.l ('2003) compiled a sample of early-type 
galaxies in the SDSS which were sufficiently bright to 
measure reliable velocity dispersions. In Fig{T2]we com- 
pare the veloc i ty dis persions of local ellipticals in the 
iBernardi et al.l (|2003f ) sample with those calculated for 
the gala xies in our sample. The stellar masses of the 
iBernardi et al.l ([2003) galaxies were calculated from their 
Ve and cry using Eq|21 We also compare to the velocity 
dispersions of late-types a nd early types ca lculated from 
the mass-size relations of (IShen et al.ll2003[ ) using Eq. O 
The quiescent and star forming galaxies fall in separate 
parts of the diagrams. The star forming galaxies roughly 
follow the local relation with a large scatter (but not 
much larger than the scatter of the local relation). The 
median velocity dispersion for a given stellar mass for the 
star forming galaxies is {cry / ay^SDSs) = 1.4 ± 0.1 times 
higher than for galaxies of similar mass in the SDSS. The 
quiescent DRGs have significantly (1.8-2.8 times) higher 
derived velocity dispersions for a given stellar mass than 
for local galaxies (median {(Ty / crv.SDSs) = 2.4 ±0.2). In- 
terestingly the quiescent DBGs do not have significantly 
higher velocity dispersions than the star forming galax- 
ies of similar mass. The highest of the inferred velocity 
dispersions of the quiescent DRGs 250— 550km/s~^ are 
higher than for any readily identifiable object in the local 



universe. Note however that possible systematic errors of 
up to a factor of two in the derived masses, could cause 
the derived velocity dispersions to be overestimated by 
a similar factor (see Sec ITQ|) . Implications of the high 
derived velocity dispersions are discussed in SeclH 

8.4. Kormendy Relationship 

In FigiT3]we plot the surface brightness of the galax- 
ies versus their size (the Kormendy relation). The rest- 
frame B-band dust corrected surface brightness of the 
galaxies has been calculated as: 

{fie)B = b^blog{7rre)^blog{2)-AB-2.blog[{l^Zph)^], 

(3) 

where b is the rest-frame B-band magnitude, derived 
from a linear i nterpolation between observed filters (see 
iRudnick et al.l 12003). re is the effective radius (in arc- 
sec), Ab is the rest- fram e B-ba nd extinction (calculated 
using the ' Calzetti et al.l (2000") extinction law assuming 
Ry = 4.05 and the best fitting Ay from the SED fit), 
and Zph is the photometric redshift. 

Also plotted is the Kormendy relation of early type 
galaxies in the SDSS (jBernardi et all I2003D . The B- 
band surface brightness is estimated by extrapolating 
the observed g and r-band surfac e brightnesses: (Mp.)^ = 
(/ie)^ + 1.97((/ie)^ - (/ie) J -0.25 CJorge usen et ld] ll995f ). 
The solid line is the best fitting relation, and the dotted 
lines are the expected relation at z = 2.5 for galaxies 
formed at z = 3,5 and 10 assuming that their surface 
brightness evolves passively from their formation redshift 
to z = (e.g assuming the slope of the local relation and 
neglecting size evolution) . The Kormendy relation of the 
z ^ 2.5 galaxies shows a large scatter but roughly follows 
a relation with similar slope as the early-type galaxies in 
SDSS but shifted to brighter surface brightnesses and 
smaller sizes. The zero point of the observed relation is 
consistent with the Zf = 5 passive evolution prediction. 
We note however that the relatively large errors and scat- 
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Fig. 12. — Left: Velocity dispersion (estimated from the size and mass) of the galaxies versus their stellar mass. Small black dots are 
measured velocity dispersions of early-type galaxies in SDSS (Bernardi et al. 2003), the curves are velocity dispersions calculated from the 
mass-size relations of Shen et ajj (|2003l) for late type (blue) and early type (red) galaxies in SDSS. The calculated relations agree well with 
the measured velocity dispersions from |Bernardi et al. (2003). Right: Calculated velocity dispersions divided by the velocity dispersion of 
late types in the SDSS (the blue line in the left plot). The star forming galaxies have velocity dispersions similar to galaxies of similar mass 
in the SDSS, but the quiescent galaxies have (0.5-3.5 times) higher derived velocity dispersions than galaxies of similar mass in the SDSS. 
Symbols are as in FigJTO] 

ter of the observed surface brightnesses makes it hard to 
put stronger constraints on the formation redshift, than 
> 4. Assuming passive luminosity evolution only, the 
Kormendy relation thus implies that the dominant stel- 
lar populations in these galaxies formed 1-2 Gyr earlier, 
when the Universe was < 10% of its present age. Lumi- 
nosity weighted formation redshifts of ~ 5 are consistent 
with ages for quiescen t DRGs derived fr om NIR spec- 
troscopy (0.5-1.5 Gyr, iKriek et al.ll2006c[ ). 

This interpretation is however too simplistic as the 
sizes likely do evolve (as shown in Sec J8.ip . Many of 
the galaxies do not have any local counterparts to evolve 
into. Simple passive evolution would result in galaxies 
which are too small and too bright for their mass. Addi- 
tional mass and size evolution (probably merger driven) 
is needed to explain the data, but the fact that the Zf = 5 
passive evolution model fits the Kormendy relation, may 
suggest that this evolution will be mainly be along the 
mass-size relation. 

9. COMPARISON TO MODELS 

We have compared the properties of the z ^ 2.5 galax- 
ies in the sample with properties of galaxies in the SDSS, 
and demonstrated that there is no simple passive evo- 
lutionary link between the two populations. To spec- 
ulate on how and when these galaxies formed and what 
they will evolve into in the local Universe (and how?), we 
must turn to models. Current self-consistent models of 
galaxy formation and evolution have a hard time repro- 
ducing the properties of DRGs, so at the moment we can 
only do a qualitative comparis on. DRGs dominat e mass 
selected samples at z ~ 2.5 ( van Dokkum et all [2006). 
The most massive galaxies are believed to form in the 
highest density regions of the universe, which are the 
first to collapse, so DRGs are good candidates for be- 




1.0 10.0 
re (kpc) 



Fig. 13. — Mean (dust extinction corrected) rest-frame B-band 
surface brightness within re, as a function of re (the Kormendy 
relation). The small points represents the Kormendy relation of 
early type galaxies in SDSS (Bernardi et al. 2003). The solid line 
is the best fit relation to the SDSS points. The dotted lines are 
the expected relation for galaxies at z = 2.5 for galaxies formed at 
Zf =3,5 and 10, which evolve passively (calculated from a single 
burst model of BC2003) from the formation redshift to 2; = (i.e 
no size evolution or change in slope of the relation). We note that 
the Zf = 5 model fit the points well, but the size distributions of 
the z ~ 2.5 and SDSS galaxies are quite different. Symbols are 
as in Fig jlOl For clarity, error bars (which are are dominated by 
uncertainties in the dust corrections) are not shown in the plot, 
but a typical size error bar is displayed in the lower left corner of 
the plot. 

ing among the first galaxies to form. Also, since the 
DRGs at z ^ 2.5 are already as massive as local ellipti- 
cals, they are likely to eventually end up as ellipticals at 
lower redshifts. But before they do, they probably have 
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to go through a number evolutionary steps, including 
major and minor mergers. As for a p ossible formation 
scena rio, semi-analytical modeling (e.g. iKhochfar fc Silkl 
l2QQ6l) suggests that elliptical galaxies formed in gas-rich 
mergers can result in very dense stellar cores, as the 
gas is driven to the center of the merger where it pro- 
duces massive starbursts. Galaxies which merge in the 
early universe are more likely to be gas-rich than galaxies 
which merge later, since they have had less time to ex- 
haust their gas reservoirs, and as a consequence, massive 
ellipticals (M > 5 x 10^^ M©) are predicted to be ~ 4 
times s maller at 2; = 2 than galaxies of similar mass at 
z = C Khochfar fc^iia [2QQ6> ) . The effect is smaller for 
less massive galaxies (M ^ 1 x 10^^ M©) which are pre- 
dicted to be ~ 35% smaller than galaxies of similar mass 
in local universe. High redshift gas-rich mergers are thus 
a likely formation scenario for the compact old quiescent 
2; ~ 2.5 DRGs. Possible examples of progenitors of quies- 
cent DRGs are sub-millimeter galaxies (SMGs) which are 
believed to be dusty starbursts in ongoing gas-rich major 
mergers. CO observations have shown that these galax- 
ies typically have dynamical masses VS^^Mc^ (mainly 
molecular g:as and stars) within 2 kpc (|Greve et al.|[2QQ5l : 
iTacconi et alll2QQ"6t iBouche et al |[2QQ7l ) , corresponding to 
large surface mass densities (though not quite as high as 
for quiescent DRGs). How the quiescent galaxies relate 
to galaxies in the local universe is unclear, but they have 
to go through processes which increase their size, with- 
out increasing their mass too much, or creating too much 
new star formation, since their observed sizes are smaller 
and their surface mass densities and derived velocity dis- 
persions are higher than for any known massive galaxy 
in the local universe. Dissipationless ("dry") merg- 
ing of gas poor s ystems could partly be responsib le for 
this process i e.g. Ivan DokkumI l2QQ5l [Nipoti et al] l2QQ3l : 
iBovlan-Kolchi n et al.l l2QQ6f K Models suggest that such 
"dry mergers" could decrease the velocity d ispersion and 
increase the size of the resulting remnant (|Nipoti et al.l 
I2QQ3I : iBovlan-Kolchin et al.l[2QQ6h . In order to reach the 
size and surface mass densities of local ellipticals the qui- 
escent galaxies would however have to go through suc- 
cessive equal mass dry mergers, ending up with masses 
> 10^^ M0. It seems unlikely that there would be that 
many progenitors of extremel y massive galaxi es in such a 
small field (see discussion in Zirm et al.ll2QQ7[ ). It is more 
likely that they undergo both minor and major mergers, 
also with more gas-rich systems, which are more abun- 
dant. Whether such mergers are able to create an evo- 
lutionary link between the z ~ 2.5 galaxies and local 
galaxies of more reasonable masses (i.e. M < 10^^ M©) 
are yet to be explored by models. 

The star forming DRGs may be easier to reconcile with 
local galaxies as they are "only" a factor of two smaller 
than galaxies with similar masses in the SDSS. Mod- 
els for the inside-out growth of disks galaxies through 
gravitational infall o f gas (e.g. Bouwens & Silk 2002; 
ISomerville et al.l 12006) could account for a factor of two 
growth in size since z ~ 2.5, but this is an unlikely sce- 
nario since the large masses of the DRGs may produce 
too many massive disk galaxies in the local universe. Ad- 
ditional processes like merging and fading of the young 
stars can also change the structure significantly. 

10. UNCERTAINTIES AND CAVEATS 



The main result in this paper is the correlation be- 
tween size and star formation activity. To investigate 
the robustness of this result we here consider the uncer- 
tainties and possible caveats of the analysis. It is evident 
from the NICMOS images that the sizes of the quies- 
cent galaxies are comparable to the size of the NICMOS 
PSF, so that sets a firm upper limit on their sizes. We 
have demonstrated that the systematic uncertainties in 
determining the sizes of the smallest galaxies are less 
than 15% (mainly due to PSF variations). One could 
argue that the NICMOS observations may fail to de- 
tect larger very low surface brightness components of the 
galaxies, but this is unlikely, as no such components were 
found for DRGs in the muc h deeper Hubble Ultra Deep 
Field (HUDF) observations (|Toft et al.ll2"QQ5h . Also, sizes 
derived from the independent very deep ground-based 
VLT+ISAAC FIRES data have been shown to agree 
well with the sizes derived here (see SecH]). Systematic 
redshift errors are perhaps the most important poten- 
tial source of uncertainty, since these are important both 
for conversion from angular size to physical size, and for 
the properties of the stellar populations derived from the 
SED fits. The faint optical magnitudes of DRGs make 
it very hard to obtain spectroscopic redshifts, so most 
of the work on DRGs (including the present) is based 
on photometric redshifts. Comparison between spectro- 
scopic and photometric redshifts suggests that the pho- 
tometric redshifts are accura te to Az/{1 + z) :^ 0.08 
(iForster Schreiber et al.|[2006f ) for galaxies with emission 
lines in the optical, and recent work based on NIR spec- 
troscopic observations of 4 DRGs suggests that the pho- 
tometric redsh ifts are also accura te for DRGs without 
emission lines (|Kriek et al.ll2006aD . The mean accuracy 
of the photometric redshifts of the four galaxies in the 
present sample with spectroscopic redshifts (see Tab [2]) 
is Az/(1 -\- z) 0.09. The important thing is that the 
number of catastrophic failures is small, so that low mass 
dusty z ^ 1 galaxies or high redshift quasars are not 
mistakenly classified as compact z ^ 2.5 massive qui- 
escent galaxies. A large number of misclassified lower 
redshift galaxies is unlikely (and a few would not change 
the conclusions), and the latter can be ruled out since 
the presence of high redshift QSOs would be evident 
from the IRAC and MIPS observations. The uncertain- 
ties in the derived quantities (physical size, mass, surface 
mass-density etc) due to photometric redshift uncertain- 
ties were estimated in the following way: We changed 
the photometric redshifts to the extreme values allowed 
by the photometric redshift error bars and repeated the 
analysis, i.e. calculated their physical size, fitted their 
SED and derived their mass and other properties from 
the best fitting models. The median error due to known 
uncertainties in the photometric redshifts is in this way 
calculated to be ~ 17% for the physiscal sizes, ~ 11% 
for the masses, ~ 15% for the velocity dispersions and 
~ 40% for the surface mass densities. 

The stellar mass of the galaxies is an important param- 
eter, both for the normalization of the size distributions 
using the local mass-size relationship, and for the inter- 
pretation of the z ^ 2.5 M - re, M - H and M - a 
relations. The most important sources of uncertainties 
in the mass estimates are the photo-z uncertainties dis- 
cussed above, and uncertainties about the validity of the 
assumed Salpeter IMF. Assuming a Kroupa IMF rather 
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than a Salpeter IMF results in masses which are smaher 
by a factor of two. Also, the present masses are de- 
rived from the best fitting BC2003 model. Using another 
stellar population synthesis model (e.g. Maraston 2005) 
could systematically shift the de rived ni asses to lower 
values (by about - 40%, see Wuvts et al. l[2007). 

As the mass distribution of the quiescent and star 
forming galaxies are similar, however, only systematic 
mass errors which are different for the two subgroups can 
change our conclusions, e.g if the masses of the quiescent 
galaxies were grossly over-estimated, and the masses of 
the star forming galaxies were under-estimated. A sys- 
tematic shift of the whole sample, e.g. to smaller masses 
may make the evolutionary trends a bit less significant, 
e.g for the size evolution and derivatives thereof (sur- 
face mass-density and velocity dispersions), but would 
not change the conclusions. 

Another possible source of systematic uncertainty is 
the possibility that selection effects prevent the detec- 
tion of large, quiescent, low surface brightness galaxies. 
It is however unlikely that a significant number of such 
galaxies were missed, given that no such galaxies was 
found in the much deeper FIRES HDFS observati ons 
(jLabbe et all 120031). in the HD FN (|Dickinsonl [2000h or 
m the HUDF ("rbft et al. "2005"). Furthermore, as shown 
bv iBouwens et al . (2004) in a study based on the UDF, 
the main effect of increasing the depth is to add galax- 
ies at fainter magnitudes, not larger sizes, demonstrating 
that high redshift galaxies are predominantly compact, 
and that large low surface brightness objects are rare. 

11. SUMMARY 

We presented HST NICMOS and ACS observations of 
41 2 < Zphot < 3.5 galaxies in the FIRES MS1054-03 
field. Combining the FIRES optical-NIR data with deep 
Spitzer IRAC data we constructed optical-MIR SEDs, 
which we fitted with synthetic stellar population models, 
to constrain the star formation histories, masses, ages 
and dust content of the galaxies. Using NICMOS im- 
ages, we performed a detailed structural analysis and 
showed that most of the DRGs are dominated by a sin- 
gle centrally symmetric component. About half of the 
DRGs are compact and half are extended. The DRGs 
are extremely faint in the optical, many of them are not 
detected, or detected at low S/N in the stacked 12h ACS 
exposure, and the ones that are detected frequently look 
very different than in the NICMOS images. We quanti- 
fied this difference by calculating the shift of the centroid 
of the light between the ACS and NICMOS images. This 
quantity was shown to depend on J-K color, with the red- 
dest galaxies having the largest shifts. The DBGs on the 
other hand are bright in the optical, and have similar 
ACS and NICMOS structure. The structural analysis 
suggests that the DRGs have composite stellar popula- 
tions with old red stars dominating the stellar mass and 
rest-frame optical structure, in some cases with substan- 
tial amounts of ongoing star formation taking place in off 
center regions of the galaxies. The DBGs on the other 
hand are consistent with simpler, uniformly younger stel- 
lar populations which dominate the structure at both 
rest-frame UV and optical wavelengths. The distribution 
of the Sersic parameters n and rg of the DRGs and DBGs 
was shown to be similar. The galaxies in both subsam- 
ples have uniformly smaller sizes than galaxies of similar 



mass in the local universe (by a factor of about 2) and the 
majority of the galaxies (~ 80%) have surface brightness 
profiles which are better fitted by "exponential disk" like 
laws, than "de-Vaucouleurs" like laws. These results are 
very similar to those deri ved from galaxies at 2 < z < 3.2 
from the FIRES survey (|Truiillo et a l. 2006). 

Next we divided the sample into "star forming" and 
"quiescent" galaxies, based on stellar population synthe- 
sis model fits to their restframe UV-NIR SEDs. The va- 
lidity of this classification was confirmed by MIPS 24/im 
observations, which showed that all except one of the 
quiescent galaxies had MIPS fluxes consistent with no 
ongoing star formation and/or AGN activity, while all 
except one of the star forming galaxies had MIPS fluxes 
consistent with significant star formation and/or AGN 
activity. 

The quiescent galaxies which constitute 22% of the 
sample, have systematically smaller sizes than the star 
forming galaxies. The star forming galaxies follow a 
mass-size relationship (with a large scatter) similar to in 
the local universe, but shifted to smaller sizes by a fac- 
tor of about 2 at a given mass. The median size of the 
quiescent galaxies is a factor of 5 smaller than galaxies 
of similar mass in the local universe, indicating a direct 
correlation between star formation activity and surface 
mass density at z ~ 2.5. 

The work presented here confirms, and adds consider- 
able confidence, to the evidence for a relation between 
star for mation activity a nd size which was found in the 
HDFS (Zirm e t al.l 12007 ^ through a much larger sample 
of galaxies (with a larger range of colors and masses), and 
the addition of MIPS 24/im data to the analysis, which 
provides independent constraints on star formation and 
AGN activity. 

The small sizes of the quiescent galaxies, combined 
with their high masses and luminosities implies that 
they have unusually high surface mass densities, surface 
brightnesses and, consequently high velocity dispersions. 
While the median surface mass density of the star form- 
ing galaxies is a factor of 6 greater than in galaxies of 
similar mass in the local universe, the surface mass den- 
sities of quiescent galaxies are more than an order of 
magnitude higher than in galaxies of similar mass in the 
local universe. The velocity dispersions derived from the 
masses and sizes of the star forming galaxies are consis- 
tent with the velocity dispersions measured for galaxies 
of similar mass in the local Universe (taking into account 
a large scatter), but for the quiescent galaxies we derive 
velocity dispersions which are a median factor of > 2 
higher. Finally we show that the Kormendy relation of 
the galaxies show a large scatter but follow a relation 
with similar slope as for early types in the local universe, 
and a zero point consistent with a mean stellar forma- 
tion redshift of Zf = 5. Combining the evidence from 
the structural and stellar population modeling analysis 
presented in this paper we conclude that in addition to 
passive aging of the stars, significant mass, size and struc- 
tural evolution is needed for the z ^ 2.5 galaxies to evolve 
into the local population. 

Our understanding of the compact quiescent galaxies 
will benefit greatly from future higher resolution NIR 
imaging studies (e.g. with WF3 on the HST) of larger 
samples, and NIR spectroscopic observations, providing 
spectroscopic redshifts, and stronger constraints on stel- 
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lar populations. 
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TABLE 1 

SED MODELS 



Id 


FIRES-Id 


SED-type 


Mass 
[lOiOM©] 


zph 


-^24//,m 

[^Jy] 


[^Jy] 


agn(«^) 


1 


1644 


Star forming 


8.41 


2.2 


59.7ib 13.4 


12. lib 0.4 


I 


2 


1712 


Star forming 


4.72 


2.0 


69.5ib 13.5 


6.6ib 0.4 




3 


1612 


Quiescent 


24.42 


2.2 


12.4± 14.2 


13.5± 0.4 




4 


1734 


Quiescent 


31.53 


2.7 


6.9± 13.6 


23. 5± 0.2 




5 


1038 


Quiescent 


4.44 


2.4 


45. 0± 19.6 


4.9± 0.4 




6 


1050 


Star forming 


6.16 


3.0 


39.0± 19.2 


3.3± 0.4 


I 


7 


1144 


Quiescent 


10.50 


2.2 


-31. 8± 13.4 


5.6± 0.4 




8 


1198 


Star forming 


23.38 


2.8 


95. 8± 15.5 


7.7± 0.4 


M 


9 


1277 


Star forming 


13.14 


2.2 


46. 5± 13.6 


6.8± 0.4 


I 


10 


1431 


Star forming 


27.37 


2.9 


212. 1± 13.6 


16.7± 0.4 


(I),M 


11 


1237 


Star forming 


5.49 


2.8 


4.5± 13.7 


4.1± 0.4 


12 


1319 


Star forming 


55.40 


2.4 


47.0± 14.1 


41. 5± 0.4 




13 


723 


Star forming 


16.90 


1.9 


411. 5± 13.7 


51. 6± 0.4 




14 


847 


Star forming 


33.12 


2.2 


275. 7± 22.9 


20. 0± 0.4 




15 


852 


Star forming 


8.25 


2.2 


300. 1± 18.1 


11. 2± 0.4 




io 




Star forming 


on QT 
zU.o / 


o n 

Z.U 


Ql a O-L OQ O 


/in T-i- n A 
4U. / it U.4 




17 


1383 


Star forming 


48.50 


2.4 


240. 9± 17.4 


42. 8± 0.4 




18 


10 


btar rormmg 


3.77 


2.9 


27. b± lo.5 






19 


b 


btar rormmg 


0.72 


o n 


oU.U± 14. b 






20 


7 


Quiescent 


14.09 


1.9 


zU.4± 15.2 






21 


64 


Quiescent 


9.63 


2.4 


1 o o_J_ ion 


3.5ih 0.3 


1 


22 


140 


Star forming 


13.83 


2.7 


25. 1± 14.4 


17. z± O.o 




23 


72 


Star forming 


23.46 


2.0 


55. 7± 16.1 


22A± 0.1 




24 


914 


Quiescent 


10.95 


2.2 


-37. 1± 14.7 


10. 9± 0.4 




25 


1061 


Star forming^^^ 


37.65 


2.1 


-3.7± 13.4 


27.1± 0.4 




26 


1100 


Star forming 


21.84 


2.9 


262. 1± 14.7 


34. 1± 0.4 


C, I, M 


27 


313 


Star forming 


31.36 


2.0 


282. 4± 13.6 


28. 9± 0.4 


C, (I) 


1 


1654 


Star forming 


1.82 


2.2 


0.0± 13.5 


1.7± 0.4 




2 


1658 


Quiescent 


2.01 


3.0 


17.0± 15.1 


O.lib 0.4 




3 


1127 


Quiescent 


0.41 


2.2 


-14. 2± 13.5 






4 


1276 


Star forming 


2.12 


3.6 


-10.6± 14.7 






5 


1253 


Star forming 


14.00 


3.0 


68.4± 15.0 


11. 6± 0.4 




6 


1227 


Star forming 


2.47 


3.6 


7.7± 13.9 


0.9± 0.4 




7 


1358 


Star forming 


2.77 


2.3 


58.2± 67.0 


6.2± 0.4 




8 


835 


Star forming 


1.67 


3.0 


24.0± 22.1 


2.5± 0.4 




9 


773 


Star forming 


2.15 


2.8 


-30.2± 14.1 


0.5± 0.4 




10 


1371 


Star forming 


3.84 


3.0 


36.0± 13.5 


0.6ib 0.4 


I 


11 


1372 


Star forming 


3.74 


2.7 


12.7± 16.4 


3.5di 0.4 




12 


1517 


Star forming 


6.90 


3.6 


-48.1± 14.5 


3.4ib 0.4 


I 


13 


5 


Star forming 


13.11 


2.0 


259.0di 15.8 






14 


1131 


Star forming 


1.63 


3.6 


0.4ib 14.6 







Note. — C) Chandra flux, I) Irac colors, M) MIPS 24/im flux consistent with 
presence of AGN. The CSF and SSP models provide equally good fits to SED, but the 
of the CSF model is marginally better. The horizontal line divides the DRG and DBG 
samples. 
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TABLE 2 
Properties of 2; > 2 sample 



Id 


FIRES-Id 






J-K 




n 




re(kpc) 


DRG-1 


1644 


2 20+2-iS 


20.85 


2.43 




1 


0.24 


1.98 


DRG-2 


1712 


1 98+°-?,5 


21.15 


2.35 




1 


0.14 


1.18 


DRG-3 


1612 


2 24+2-2^ 


20.15 


3.08 




2 


0.23 


1.93 


DRG-4 


1734 


1 

— 0.04 


19.49 


2.40 


2.697 


1 


0.12 


0.92 


DRG-5 


1038 




21.35 


2.50 




1 


< 0.08 


0.61 


DRG-6 


1050 


^•^'^ — 0.14 


21.42 


3.07 




4 


0.93 


1.36 


DRG-7 


1144 


0.10 


20.70 


3.15 




1 


0.10 


0.81 


DRG-8 


1198 


2 82+2-1^ 

— 0.14 


21.60 


3.68 




1 


0.08 


0.61 


DRG-9 


1277 


^•^^ — 0.16 


21.29 


3.11 




4 


0.13 


1.05 


DRG-10 


1431 


9 Q9+0.50 
^•^^ — 0.24 


20.99 


3.06 




2 


0.41 


3.22 


DRG-11 


1237 


9 09+0.12 
^•"^ — 0.20 


21.11 


2.66 




1 


0.14 


1.06 


DRG-12 


1319 


2 54+2-9S 


19.02 


2.59 


2.424 


3 


0.37 


3.02 


DRG-13 


723 




19.46 


2.50 




2 


0.65 


5.46 


DRG-14 


847 


9 94+0.56 
^•^^ — 0.18 


20.32 


2.98 




1 


0.20 


1.61 


DRG-15 


852 


9 99+0.74 
z,.z,z,_Q_24 


20.95 


2.56 




1 


0.27 


2.19 


DRG-16 


926 


1 98+2-2^ 


20.32 


2.71 




1 


0.34 


2.84 


DRG-1 7 


1383 


2 26 +2-?5 


19.56 


2.89 


2.423 


4 


1.26 


10.26 


DRG-18 


10 


9 QQ+0.06 

Z.OO n no 
— O.Oo 


21.56 


3.39 




2 


0.10 


0.76 


DRG-19 


6 


2 00 

-^.uu _o.06 


20.66 


2.47 




1 


0.55 


4.63 


DRG-20 


7 


1.86 +°-°^ 

x.uvj —0.06 


20.58 


2.47 




1 


< 0.08 


0.63 


DRG-21 


64 


2 42 +2-^0 

^•^^ —0.06 


21.56 


3.88 




2 


< 0.08 


0.61 


DRG-22 


140 


2 84+2-?,2 

^•^^ — 0.06 


19.77 


2.69 


2.705 


1 


0.09 


0.72 


DRG-23 


72 


2.02 +i-no 

^.vj^ —0.02 


20.20 


2.74 




1 


0.39 


3.00 


DRG-24 


914 


r, 99 + 0.14 
^•^^ — 0.08 


19.81 


2.32 




1 


< 0.08 


0.62 


DRG-25 


1061 


9 -,9+0.12 


19.63 


3.04 




3 


0.19 


1.55 


DRG-26 


1100 


9 QQ + 0.06 

^.UU>_Q Qg 


20.65 


2.34 




1 


1.04 


8.09 


DRG-27 


313 


2 02+2-^^ 

^•^^ — 0.02 


20.18 


2.79 




3 


0.88 


7.36 


DBG-1 


1654 


2 22+2-';;^ 

^•^^ — 0.06 


21.67 


2.27 




1 


0.12 


0.99 


DBG-2 


1658 


3 04+2-^2 


21.12 


2.29 




1 


0.11 


0.83 


DBG-3 


1127 


^•-^"-0.18 


21.67 


1.39 




1 


0.10 


0.79 


DBG-4 


1276 


O.UU_Q 22 


21.40 


1.72 




2 


0.12 


0.86 


DBG-5 


1253 


3.00+°-24 

— U. (\) 


20.19 


2.05 




2 


0.99 


7.66 


DBG-6 


1227 


Q ^.+0.12 

3.64^028 


21.64 


1.92 




1 


0.16 


1.12 


DBG-7 


1358 


9 90+0.10 
^•^^-0.12 


21.26 


2.23 




1 


0.18 


1.45 


DBG-8 


835 


o.UU_Q Q4 


21.66 


1.66 




2 


0.20 


1.52 


DBG-9 


773 


9 7^+0.04 
^- '"-0.48 


20.99 


1.46 




1 


0.09 


0.73 


DBG-10 


1371 


2.98ir6'o 


21.52 


1.99 




1 


0.73 


5.63 


DBG-11 


1372 


2.68i°o:ll 


21.23 


2.28 




1 


0.80 


6.37 


DBG-12 


1517 




21.42 


1.67 




4 


< 0.08 


0.54 


DBG-13 


5 


9 09+0. 06 
^•'-'^-0.08 
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Note. — Photometric redshift errors are derived from the full photometric 
redshift probability distribution P(z) as computed using a Monte-Carlo simulation 
(see Rudnick et al. 2003). P(z) can be highly non-Gaussian, due to the complex 
dependence of flux errors and template type on the allowable redshift range. For 
this reason the 68% confldence intervals for some of the sources are highly non- 
symmetric. K-band (Vega) "total" magnitudes, corrected for Galactic extinction. 
(•^^ J-K (Vega) colors derived in colour apertures, corrected for Galactic extinction. 
The horizontal line divides the DRG and DBG samples. 



